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ABSTRACT 


In previous work, the authors embedded a neural extended Kalman filter algorithm 
in a closed loop feedback system and presented results on a servo-tracking 
problem. The reason behind using a neural extended Kalman filter algorithm was 
to improve the estimator’s mathematical model. Without the correct model the 
unmodeled dynamics or nonlinearities in the system would yield an unstable filter. 


In this paper, a neural extended Kalman filter algorithm is used for tracking of a 
highly maneuvering target. The neural extended Kalman filter is used to improve a 
mathematical motion model for use in prediction. Instead of just applying a high 
process noise model (a catch all technique) in an interacting multiple model 
architecture to hold a target through a maneuver, a neural extended Kalman filter is 
used to predict the correct velocity and acceleration states of a target. This in turn 
may allow noise reduction during a target maneuver. 


Tracking results that stress the algorithm during severe maneuvers will be shown 
along with the tuning parameter issues of the artificial neural network. 
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1 . 


Introduction 


The Robust Tracking with a Neural Extended Kalman Filter (NEKF) project is an Office of Naval 
Research (ONR) In-House Laboratory Independent Research (ILIR) sponsored effort at SPA WAR 
Systems Center San Diego [1]. The project’s goal is to provide an improved state estimation capability 
for current U.S. Navy tracking systems. The NEKF will provide added capability for the real-time 
modeling of maneuvers and, therefore, will enhance the ability of tracking systems to adapt appropriately. 


The NEKF has been used in the past in control system technology and for system identification [2, 3, 4, 5, 
6, 7, and 8]. During this project the NEKF will be incorporated into a tracking architecture to provide 
robust tracking capabilities that are currently unavailable. 

2. Background 


State estimation and tracking of highly maneuvering targets lias and still is an extremely difficult task in 
modern tracking systems. Current state estimation approaches to the tracking problem include alpha-beta 
filters, Kalman filters, interacting multiple model (IMM) filters, probabilistic data association (PDA) 
trackers, joint PDA (JPDA) trackers to name a few [9 and 10]. State estimation is the problem of deriving 
a set of system states that are of interest to a system or decision maker. System states consist of 
parameters such as position, velocity, frequencies, magnetic moments, and other attributes of interest. 
Most often system states are not measurable at the system output. For example, range and bearing of a 
target may be available from a radar sensor but the position and velocity of the target needs to be derived 
from the radar measurement. To derive these states an estimation algorithm is used. A mathematical 
system model is necessary for the aforementioned filter algorithms to perform state estimation. 


A well known state estimation algorithm is the tried and true Kalman filter developed four decades ago by 
R. E. Kalman [11]. The Kalman filter is widely used in government and industry tracking problems. The 
Kalman filter uses an assumed mathematical system model (i.c. a straight line motion model for an 
aircraft) to estimate the states (position, velocity, signatures, etc...) of the aircraft. A problem occurs 
when the aircraft or system being tracked changes from the assumed motion model. The filter will tend to 
lag behind the true state of the target or even diverge and become unstable and unable to estimate the 
system. A block diagram of the Kalman filter is shown in Figure 1 below. The filter consists of the 

dynamic system to be tracked, a mathematical system model F(^), an observation model H ( k +1), the 
Kalman gain K (k +1), a predicted observation z (k +1), and the system state vector x(£ | k) . The 
Kalman filter is a recursive estimation algorithm that is driven by the observation data z(k + \) from the 

dynamic system. As data is processed a filtered estimate of the state x(& | k) is generated by the filter. 

As long as the dynamic system is linear, and matches the assumed motion model, the filter will perform 
quite well and tracking will be successful. In cases where motion model and/or the observation model are 
nonlinear, then an extension of the linear Kalman filter must be used. An extension of the Kalman filter 
is the extended Kalman filter (EKF) [12]. The EKF is used to handle known nonlinearities. The extended 
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Kalman filter evaluates the nonlinear system model f about the current state and the nonlinear 

observation model h ( x(& + 1), k + 1 ) about the predicted state and II (k + 1) is the Jacobian of the 

nonlinear observation equations evaluated at the predicted state. The extended Kalman filter equations 
are shown below in Figure 2. The Kalman filter is the same except for the nonlinear equations mentioned 
previously. 


x (k +11 k +1) 



Figure 1. Kalman Filter Block Diagram 


System Model 

x(A + l) = f[x(A),A]+L[x(A),A]w(A) 
•/.(A + l) = h[x(A + l),A + l]+v(A + l) 


Update Equations 

x(A + l|A) = f[x(A|A),A] 

P(A + 1|A) = A(A)P(A|A)A'(A)+L(A)Q(A)L'(A) 


Initialization 

x (010) =/•[*(())] 
P(0|0)=P(0) 


Kalman Filter Gain 

K(A + I) = P(A + I|A)H'(A+I)[h(A + I)P(A + I|A)H'(A + I)+R(A + I)J‘ 
where 

R(A + l)= fi[v(A + l)v'(A + l)] 


Prediction Equations 

x(A +11 A) = f [x(A | A),A] 

P(A + I|A) = A(A)P(A|A)A'(A)+L(A)Q(*)L'(A) 
where 


Q(A)-£[w(A)w'(A)] 

1 0x(A) 

L(A) = L[x(A|A),A] 


Updated Covariance 

P(A + I |A + I)=[l-K(A + I)H(A + I)]P(A +11 A) 


Updated State 

x(A + l |A + l) = x(A + l| A)+K(A + l)Az(A +11 A) 
Az(A + 11 A) = z(A + l)-h[x(A +11 A),A +1] 


Figure 2. Extended Kalman Filter Equations 
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Another well known state of the art tracking technique is the interacting multiple model (IMM) filter [13, 
14, 15, and 16]. The technique employs multiple models (a hank of Kalman filters) to perform state 
estimation. Each model may contain a different mathematical system model, observation model, variable 
dimensional state, or noise processes. For example, an IMM filter could consist of two Kalman filters, the 
first model using a straight line motion model with a low process noise and a second model with a high 
process noise. The two models are mixed probabilistically to fit the dynamics of the target. As long as the 
target follows a straight line motion and does not deviate from it, the first model will be in effect. As a 
target maneuvers from a straight line motion, the second model becomes in effect and in a sense the noisy 
positional measurements are connected together yielding a very poor velocity estimate of the target during 
the maneuver. The IMM architecture can also be used with an EKF used as one of its models. As before, 
to use the EKF we must know the nonlinearity of the system. When it comes to tracking moving targets 
we do not know what the craft will necessarily do next. This causes a problem with trying to model or 
account for a good motion model for the filter to use. 

If a nonlinear model is unattainable, than a system identification technique could be used to create a 
model. A big buzz in the late 80’s early 90’s was artificial neural networks. A neural network is a 
function approximation technique. Given a set of inputs and a desired set of outputs a neural network 
could be trained to approximate well any function. The Stone Weierstrauss theorem states that given the 
class of squashing functions, we can uniformly approximate any continuous function [17]. This is purely 
an existence theorem. It does not state how to construct a neural network or how to train its weights. 
Neural networks contain a set of weights that must be determined to approximate functions. To train 
neural networks, techniques such as backpropagation [18, 19], evolutionary programming [ 20 ], and the 
extended Kalman filter [21] have been used. In [21], Singhal & Wu asked the question, “Why not use a 
Kalman filter to estimate the weights (states) of a neural network?” What they hypothesized was that the 
weights of a neural network are also the states of the network. The EKF training technique trains 
networks on the order of tens of iterations instead of thousands of iterations for such methods as 
backpropagation. An artificial network equation is shown in equation 1 . 


Equation 1. 



i -1 v V *=i 



where fi = — --— 7 -r is the output of the ith hidden node and Xi is the dot product sum of the 

1 + exp(-*i) 

previous input layer’s output with the connecting weights of the hidden layer, NNm is the mth output of 
the neural network, Wim is the mth output weight connected to the ith hidden node, Wkt is the kth input 
weight connected to the ith hidden node, and Ik is the kth input feeding the neural network. Stubberud 
took the EKF trainer one step further. 

3. Neural Extended Kalman Filter 


The Neural Extended Kalman Filter (NEKF) is based on the Singhal and Wu EKF neural network trainer 
Stubberud took the technique one step further. Basically the algorithm is to use a Kalman filter to 
estimate the states by using a linear system model and at the same time use the Kalman filter to train a 
neural network to calculate the (nonlinearities, mismodeled dynamics, higher order modes, etc...) of the 
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system. Estimation of the system states are performed at once without the necessity of modeling the 
nonlinearities a priori as in the case of the Extended Kalman Filter. The NEKF architecture is shown in 
Figure 3. The inputs to the neural network are the updated states of the filter. The inputs are passed 
through an input layer, a hidden layer with nonlinear squashing functions, and an output layer. The 
outputs of the neural network are nonlinear corrections to the linear predicted state of the Kalman filter. 
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Figure 3. NFKF Architecture 
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The following are the NEKF Equations including the calculation of the Jacobian with respect to the neural 
network. This network is specifically for a 4 hidden node network with four states. 

3.1. Predictor Equations for the NEKF 

x(k + 1 / k) = x(k / A) + T s v x (A / A) + AW1 (x(A/ A), \v(A / k )) 
v r (A + 1 Ik) = v x (A /A) + NN2(x(kl A), w(A /A)) 

J< A + 1 Ik) = jik/k) + T x v y (k Ik) + AW3(x(A/ A), w(A / A)) 
v v (A + l/A)= v v (A/A)+AW4(x(A/A),w(A/A)) 

w(A +1 /A)= w(A /A) 
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3.2. Neural Network Equations in NEKF 


AWl(x(A/A),w(k/k)) = Y, w h 4 *)) T Uk/k) + w} 5 y w } 5 

7=1 

4 

AW2(x(A/A),\v(k/k)) = ^ ffavj) T i(k /A) + iv 1 ^H’25 

7=1 

4 

AW3(x(A /A-), w(k/ k)) = ^ wy f(^w \) 7 x(A/ A) + vvj- 5 ^-t-11-35 

7=1 

4 

AW4(x(A/A),w(k/k)) = /((wJf) r x(A/A) + ivj 5 y W45 

7=1 

where 

(W/) r = [vj| w ) 2 wj 3 vt-J'4 ] ; 7 = 1,2,3,4 

x(k/k) = [vfA/A) v v (A/A) y(A/A) v/A/A)] 


3.3. The NEKF State Vector 

Define 



0 o ~i 

>«;/3 ”74J; 7 = 1.2,3,'4 

1 1 1 

”35 ”’45 J 

2 2 1 
”'35 ”’45 J 


\ T 7 7 j t t t 

Note that the notation forCw^) and (w^) is different from that for (w •) and (w J ) 1 for 


The NEKF state vector can he written as 


= 1,2,3,4. 
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\(k!k) = 


r x(^/^)i 
!*(*/*> J 


wliere 


XV = 


r wj i 

w 2 

' v 3 

w\ 

w< 

w? 

w 2 

W 3 

W 4 

UiJ 


Note that w], w]>, w{, and represent 16 weights in the hidden layer of the neural network, wj. represents 
4 biases in the hidden layer of the neural network, wf, w$, W 3 , and wj represent 16 weights in the output layer 
of the neural network, and W 5 represents 4 biases in the output layer of the neural network. 

Thus the vector w has 40 elements and the state vector of the NEKF has 44 elements. 

Note that the prediction vector is given by 


x(/c + l//c) = 


|~ x( k -T 1 / k )~] 
[ xv(k + 1/ £)J 


3.4. The Jacobian Matrix 


The Jacobian matrix J is a 44 x44 matrix generated by differentiating the vector x(k + \/k) by the vector x(k/k), that is, 
_ Mk + \/k) 
ck(kIk) 
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We now make the following definitions 

rVwhn ftw? 


Wj 

2 1 

1 \V Z - 1 

\ f 2j\ 

(w 2 ) 

( w 3> r 

W -| 

, 2,7’ 
O 3 ) 

_(W4) 7 


, 2 sT 

l(w 4 ) J 


/(*/)- 


df(z) 


& 


where zj = (vv j ) ‘ x(k / k) + vvj 5 


2=2 j 


F(z) = 


r/(z,) 0 0 0 1 

I 0 f(z 2 ) 0 0 | 

I 0 0 /(z 3 ) 0 

0 0 0 f(z 4 ) J 


z = [z, z 2 z 3 z 4 ] 


/(Z;) = /(Z1 


f r (z) = [/(z,) /(z 2 ) /(z 3 ) /(z 4 )] 


rf r (z) O 7 0 r 0 7 1 

FW-j f | 

1 0 7 0 7 f 7 (z) 0 7 1 


o 7 o 7 


O 7 f 7 (z) 


where 0 7 = [0 0 0 0] 


Ti 7 (/://:) O ' 


X(k/k) = \ 


0 7 \ T (k/k) o 7 

n T J «T 


o 7 1 


0 ' 


x (k/k) 0' 


0 ' 


- T 


x(k/k) J 
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0 „ X/ „ =zero matrix with n rows and m columns 

I ;)X;) = identity matrix with n rows and columns 

Using these definitions, the Jaeobian matrix ean be written as 

W 2 F( 7 .)X(k/k) | W 2 F(z) | F(z) j 

1 40x40 


I A+ W 2 F(z)W’ 


O 


40x4 


4. Tracking result comparisons 


Tlie following is a set of comparison results between a linear Kalman filter, a linear IMM 2 model filter, 
and two NEKF’s with 4 and 10 hidden nodes, respectively. To make the comparisons between the 
tracking filters each filter used the same amount of proeess noise, except for the IMM where its first 
model used none. Each filter used a constant velocity straight line motion model with a 4 state vector 
consisting of position and velocity in a Cartesian coordinate system. The neural network architecture for 
both networks only used 2 outputs in this simulation. These two outputs were nonlinear or unmodeled 
higher order mode eorreetions to the velocity states. The nonlinear squashing functions used by the 
neural networks here hyperbolie tangent functions. The network with 4 hidden nodes had a total of 30 
weights to be trained at eaeh time sample for adaptation and the network with 10 hidden nodes had 72 
weights. This tracking study was for a single target on a two dimensional Cartesian grid. A jerk motion 
model was used to generate truth. This was used to yield a smoother ramp up for acceleration and 
velocity for a more realistic scenario. Figure 4 shows the true position of the track for the scenario. The 
track starts at (0,0) in the grid at a speed of 10 units/time sample along the x-axis. It then increases its 
speed to 35 units/time sample and then decelerates back to 10. Then the traek turns right and aeeelerates 
through a 180 degree turn. Its top speed though the turn is 90 units/time sample. The traek then 
decelerates baek to 26 units/time sample. A left 180 degree turn is then performed moving the traek baek 
on the positive x-axis direction. The track then decelerates again to 20 units/time sample. Finally, the 
traek moves into five 360 degree turns before the scenario ends. A radar equation was implemented for 
the simulation to generate range and bearing measurements from the sensor to the target. Gaussian noise 
was added to both the true range and true bearing to be used by eaeh filter. The amount of uncertainty 
used varied from 10, 50, and 100 units of noise for range and 1,2, 3, and 10 degrees for bearing. The 
sensor location in this scenario is loeated at the origin. The sensor location eaused the beginning data of 
the target to be very noisy due to the measurements all falling very elose into the sensor. As the target’s 
trajectory moved away from the origin the data began to look more like the trajectory and not just 
bouneing around aimlessly. Figure 5 is shown to provide an idea of noisy range and bearing 
measurements. They have been converted to x,y coordinates for display purposes. In the figure a Monte- 
Carlo Kalman filter result is overlayed in blue. Notice the large lag when the aeeelerating turn oecurs and 
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Figure 4. Scenario 
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Figure 5. Noisy Observations 
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when the constant velocity turns occur. Figure 6 is a zoomed in view of the 5 circular turns of the target 
that look like a hornet’s nest. This figure shows another result from the Kalman filter. The range/bearing 
error on the measurements can be seen in this figure. Notice the right and left sides of the track lag 
heavily and the top and bottom track rather well close to truth. This is due to the target trajectory in 
reference to the sensor location. Figures 7-9 are results from a typical network run. Figure 7 shows the 
weights of a four hidden node neural network. The number of weights in the neural network are 30 for 
this network. The large swings in network adaptation arc due to the target maneuvers causing the 
network to add nonlinear corrections to the output of the linear Kalman filter. Of note, the neural network 
is also adding unmodeled mode dynamics such as acceleration corrections or jerk corrections to adjust the 
velocity states. For example, if in the real target track an acceleration is increasing the velocity in the x 
direction by two, the neural network will adapt and increase the velocity estimate by two units. The 
neural network adapts through the residual error between the predicted observations and the noisy 
observations from the sensor. In Figure 7 the first real large 


Single Run Target Position Estimate 



X-Axis 


Figure 6. Hornet’s Nest 


Adaptation occurs at time 50. This is when the first acceleration is performed by the target. At time 80 
the target decelerates back to its original speed and the network adapts again. At time 120 a very large 
adaptation occurs in the neural network. At this time, the target is accelerating and performing a 180 
degree turn. The speed increases by 9 times by the end of this maneuver from 10 units/time sample to 90 
units/time sample. The network has to keep adjusting as each data point is processed to keep up with the 
large maneuver. At time 170 the target then decelerates back to a speed of 26 units/time sample. At time 
230 the second 180 degree turn occurs and the network adapts again. At time 275 another deceleration 


































occurs and the network adapts again. Finally, at time 340 until the end of the scenario the target’s 
trajectory is a constant speed 5 degree/time sample turn. From this time on in the figure it can be seen 
that the neural network weights are adapting in a sinusoidal fashion to accommodate the corrections to the 
x and y velocity states. Another note about Figure 7 is the weights that almost look constant throughout 
the run. These weights are the bias weights in the network’s input and output layers. These weights 
perform noticeable adjustments when strong maneuvers occur in the trajectory. These bias weights are 
needed to perform an affine transformation to move the function approximation from having to exist 
always at the origin. A final note about this figure is since the observations are noisy the weights are 
always constantly adapting. The sinusoidal inputs look noisy because of this. The better the sensor the 


Neural Network Weights 



Figure 7. NEKF Weights 


less the weights have chatter in their adaptation. Figures 8 and 9 are the outputs for the y and x velocity 
corrections for the neural network, respectively. Recalling that the first maneuvers at time 50 and 85 only 
occur in the x-direction, Figure 8 shows that the corrections for the y-velocity state are very small and in 
the noise, figure 9, however, contains adjustments for the x-velocity where the maneuvers occur. At 
these times the network increases the x-velocity and then decreases when the target slows down again. 
When the accelerating turn occurs at time 120 the network’s outputs for x and y velocity corrections are 
the largest. When the maneuvers stop and the trajectory returns to straight line motion the network’s 
outputs go to small levels. Notice at the end of the figures the sinusoidal corrections for adapting through 
the circular motion. 


12 






































NN Y Velocity Estimate 


Neural Network Output Vy 



Fimire 8. Y Velocity Correction 

Neural Network Output Vx 



Figure 9. X Velocity Correction 
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For each of the tracking results in Figures 10-15 a time average position error statistic of 50 Monte-Carlo 
runs was used to generate the plots. The statistic was calculated by the following four equations 

pos err - pos err + yj(\-xtnith)' +(y -ytruthy 

noise_err = noiseerr + -xtruth)~ +(z v — ytruih ) 

Delta = noise err-pos error 

avg(i) - avg(i+\) + Delta(i) 


where pos_err is the position error between the state estimates and truth, noise__err is the position error 
between the noisy observations and truth, Delta is the difference between the noise error and the estimate 
error, and avg is the time average of the error statistic. If the avg is increasing the filter is doing well, 
and if the avg is decreasing the filter is performing poorly. Fifty Monte-Carlo runs were averaged for 
each error statistic that is plotted in Figures 10-15. In each figure all four tracking filters are plotted 
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Figure 10. R = 50, B = 2 


together and overlayed on the same plot. Each plot has a legend where the network with 10 hidden nodes 
is blue, the network with 4 hidden nodes is red, the Kalman filter is magenta, and the IMM is green. 

Figure 10 is a result where the range uncertainty was set to 50 units, and the bearing uncertainty to 2 
degrees. In this figure the IMM is ahead of the NEKF results until the hornet’s nest maneuver. Then both 
NEKF’s perform much better than both the Kalman filter and the IMM. In Figure 11, the bearing 
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uncertainty is now set to 1 degree. The NEKF’s once again outperform the IMM and the Kalman filter. 
In Figure 12 the range uncertainty was increased to 100 units. In this case the NEKF’s performed the 
worst compared to the Kalman filter and the IMM. This was due to the initial measurements from the 
sensor having large amounts of noise and not allowing a trajectory to form until the target moved far 
enough away from the origin. It is believed that if the scenario would have run longer that the NEKF’s 
would have caught up with the other estimators. Notice at the end of the error statistic plots on Figure 12 
that the NEKF results are increasing and the IMM and Kalman filter are starting to decrease. A box car 
average has been suggested to better compare the different estimators. 
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Figure ll.R = 50 ,B = 1 


Figure 13 shows the results of setting the range uncertainty to 10 units along with the bearing uncertainty 
still set to 1 degree. This plot more clearly shows the NEKF’s outperforming the other two filters. Figure 
14 shows the results of setting the range uncertainty to 100 units with the bearing uncertainty set to 3 
degrees. Once again in this figure the NEKF’s outperform the other two trackers. Finally, in Figure 15 
the results of setting the range uncertainty to 100 units with the bearing uncertainty set to 10 degrees is 
shown. This figure shows the NEKF’s lower in score, but all of the filter results are very close together in 
this highly noisy case. 
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Figure 12. R = 100, B = 1 



Figure 13. R = 10, B = 1 
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Figure 14. R= 100,13 = 3 
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Figure 15. R = 100, B= 10 
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5. Research Issues 


Some of the research issues this project is trying to address is as follows. 

• Show that the Neural Extended Kalman Filter can he embedded into an Interacting Multiple 
Model (NEKF IMM) algorithm 

• Show that the NEKF IMM algorithm provides smoothed estimates during system dynamics 

• Show that the NEKF provides smoothed estimates during system dynamics 

• Show that the NEKF has a second order convergence 

• Develop techniques for choice of optimal noise processes of the NEKF 

• Determine the neural network architecture needed to work across a broad range of problems 

• Show improved tracking performance over other techniques (alpha-beta, Kalman, 1EKF, IMM, 
etc...) 

• Investigate the effects of the NEKF in the IMM architecture 

• Targeting specific states (velocity) or is all more robust 

• Observability dependence on initial conditions 

• Observability can lead to a Stability Proof 

• Multi-Target Tracking 

• Data Association 

• Pd, Pfa, Loss of Signal Detection making a more complex realistic scenario 

• Squashing Functions 

• Multi-Layer Networks 

• Is an IMM architecture required 

• Tracking Benchmark Study 

• Higher Order Models 

• Data Fusion in a Multi-Sensor Environment 

6. Conclusions 

In this paper we discussed the neural extended Kalman filter as applied to the target tracking problem 
The NEKF uses a neural network to adapt on-line to unmodeled dynamics or nonlinearities in the target 
trajectory. This on-line adaptation provides for a robust state estimation for tracking applications because 
the maneuvers do not have to be known beforehand. The NEKF is a generic state estimator that can be 


18 










used to estimate any state vector such as position, velocity, magnetic moment, frequency signatures, etc... 
Comparisons were performed between a linear Kalman filter, a 2 model IMM filter, and two NEKF’s 
with differing hidden layers. The NEKF outperformed the other filters in almost every case analyzed. 
Further investigation into why the NEKF did not outperform in each case is now under investigation. 
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